Streaming an image through the eye: The retina seen as a dithered scalable image coder

We propose the design of an original scalable image coder/decoder that is inspired from the mammalians retina. Our coder accounts for the time-dependent and also nondeterministic behavior of the actual retina. The present work brings two main contributions: As a first step, (i) we design a deterministic image coder mimicking most of the retinal processing stages and then (ii) we introduce a retinal noise in the coding process, that we model here as a dither signal, to gain interesting perceptual features. Regarding our first contribution, our main source of inspiration will be the biologically plausible model of the retina called Virtual Retina. The main novelty of this coder is to show that the time-dependent behavior of the retina cells could ensure, in an implicit way, scalability and bit allocation. Regarding our second contribution, we reconsider the inner layers of the retina. We emit a possible interpretation for the non-determinism observed by neurophysiologists in their output. For this sake, we model the retinal noise that occurs in these layers by a dither signal. The dithering process that we propose adds several interesting features to our image coder. The dither noise whitens the reconstruction error and decorrelates it from the input stimuli. Furthermore, integrating the dither noise in our coder allows a faster recognition of the fine details of the image during the decoding process. Our present paper goal is twofold. First, we aim at mimicking as closely as possible the retina for the design of a novel image coder while keeping encouraging performances. Second, we bring a new insight concerning the non-deterministic behavior of the retina.Comment: arXiv admin note: substantial text overlap with arXiv:1104.155

Harris Corners in the Real World: A Principled Selection Criterion for Interest Points Based on Ecological Statistics

In this report, we consider whether statistical regularities in natural images might be exploited to provide an improved selection criterion for interest points. One approach that has been particularly influential in this domain, is the Harris corner detector. The impetus for the selection criterion for Harris corners, proposed in early work and which remains in use to this day, is based on an intuitive mathematical definition constrained by the need for computational parsimony. In this report, we revisit this selection criterion free of the computational constraints that existed 20 years ago, and also importantly, taking advantage of the regularities observed in natural image statistics. Based on the motivating factors of stability and richness of structure, a selection threshold for Harris corners is proposed that is optimal with respect to the structure observed in natural images. Following the protocol proposed by Mikolajczyk et al. \cite{miko2005} we demonstrate that the proposed approach produces interest points that are more stable across various image deformations and are more distinctive resulting in improved matching scores. Finally, the proposal may be shown to generalize to provide an improved selection criterion for other types of interest points. As a whole, the report affords an improved selection criterion for Harris corners which might foreseeably benefit any system that employs Harris corners as a constituent component, and additionally presents a general strategy for the selection of interest points based on any measure of local image structure

New algorithm for solving variational problems in W^{1,p}\SO and BV\SO: Application to image restoration

We propose a new unifying method for solving variational problems defined on the Sobolev spaces $W^{1,p}(\Omega)$ or on the space of functions of bounded variations $BV(\Omega)$ ($\Omega\subset\R^N$). The method is based on a recent new characterization of these spaces by Bourgain, Brezis and Mironescu (2001), where norms can be approximated by a sequence of integral operators involving a differential quotient and a suitable sequence of radial mollifiers. We use this characterization to define a variational formulation, for which existence, uniqueness and convergence of the solution is proved. The proposed approximation is valid for any $p$ and does not depend on the attach term. Implementation details are given and we show examples on the image restoration problem

Biological model of motion integration and segmentation based on form cues

Active vision is an essential part of every biological organism possessing an eye system: posture, eye movements, visual research, ... All require a motion percept to operate. At the basis of active vision lays the ability to calculate movements of objects in the scene, at least on a sufficient level to react correctly. In this report we present a model of motion integration and segmentation in the first visual cortex areas. Specifically we modeled the first two cortex areas involved in motion processing in the primate: V1 and MT. To be able to process motion correctly a visual system also need to deal with form information. We investigate how form cues coming from the ventral pathway can be used by the V1/MT dorsal pathway to solve some perception problems. By using a recurrent dynamical system between the V1 and MT layers we are able to find out psychophysical results such as motion integration and center-surround effects due to the feedback connections, or end-of-line and 2D features detectors thanks to the shunting inhibition. We propose to modulate this system by a form information coming from the ventral stream and are thus able to explain asymmetric center-surround effects as well as motion segmentation and segregation between extrinsic and intrinsic junctions

Virtual Retina : a biological retina model and simulator, with contrast gain control

A detailed retina model is proposed, that transforms a video sequence into a set of spike trains, as those emitted by retinal ganglion cells. It includes a linear model of filtering in the Outer Plexiform Layer (OPL), a contrast gain control mechanism modeling the non-linear feedback of some amacrine cells on bipolar cells, and a spike generation process modeling ganglion cells. A strength of the model is that each of its features can be associated to a precise physiological signification and location. The resulting retina model can simulate physiological recordings on mammalian retinas, including such non-linearities as cat Y cells, or contrast gain control. Furthermore, the model has been implemented on a large-scale simulator that can emulate the spikes of up to 100,000 neurons

Experimenting the variational definition of neural map computation

International audienc

Could early visual processes be sufficient to label motions?

Biological motion recognition refers to our ability to recognize a scene (motion or movement) based on the evolution of a limited number of points acquired for instance with a motion capture tool. Much work has been done in this direction showing how it is possible to recognize actions based on these points. Following the reference work of Giese and Poggio (giese-poggio:03), we propose an approach to extract such points from a video based on spiking neural networks with rank order coding. Using this estimated set of points, we verify that correct biological motion classification can be perfomed. We use some recent results of Thorpe et al. (rullen-thorpe:01,thorpe-fabre-thorpe:01,delorme-perrinet-etal:01) who claim that the neural information is coded by the relative order in which these neurons fire. This allows to select a limited set of relevant points to be used in the motion classification. Several experiments and comparisons with previous neurological work and models are proposed. The result of these simulations show that information from early visual processes appears to be sufficient to classify biological motion

Motion integration modulated by form information

ISBN : 978-2-9532965-0-1We propose a model of motion integration modulated by form information, inspired by neurobiological data. Our dynamical system models several key features of the motion processing stream in primate visual cortex. Thanks to a multi-layer architecture incorporating both feedforward-feedback and inhibitive lateral connections, our model is able to solve local motion ambiguities. One important feature of our model is to propose an anitropic integration of motion based on the form information. Our model can be implemented efficiently on GPU and we show its properties on classical psychophysical examples. First, a simple read-out allows us to reproduce the dynamics of ocular following for a moving bar stimulus. Second, we show how our models able to discriminate between extrinsic and intrinsic junctions present in the chopstick illusion. Finally, we show some promising results on real videos

Modelling the dynamics of motion integration with a new luminance-gated diffusion mechanism

The dynamics of motion integration show striking similarities when observed at neuronal, psychophysical, and oculomotor levels. Based on the inter-relation and complementary insights given by those dynamics, our goal was to test how basic mechanisms of dynamical cortical processing can be incorporated in a dynamical model to solve several aspects of 2D motion integration and segmentation. Our model is inspired by the hierarchical processing stages of the primate visual cortex: we describe the interactions between several layers processing local motion and form information through feedforward, feedback, and inhibitive lateral connections. Also, following perceptual studies concerning contour integration and physiological studies of receptive fields, we postulate that motion estimation takes advantage of another low level cue, which is luminance smoothness along edges or surfaces, in order to gate recurrent motion diffusion. With such a model, we successfully reproduced the temporal dynamics of motion integration on a wide range of simple motion stimuli: line segments, rotating ellipses, plaids, and barber poles. Furthermore, we showed that the proposed computational rule of luminance-gated diffusion of motion information is sufficient to explain a large set of contextual modulations of motion integration and segmentation in more elaborated stimuli such as chopstick illusions, simulated aperture problems, or rotating diamonds. As a whole, in this paper we proposed a new basal luminance-driven motion integration mechanism as an alternative to less parsimonious models, we carefully investigated the dynamics of motion integration, and we established a distinction between simple and complex stimuli according to the kind of information required to solve their ambiguities